Non-Uniform Time-Step Deep Q-Network for Carrier-Sense Multiple Access in Heterogeneous Wireless Networks

This paper investigates a new class of carrier-sense multiple access (CSMA) protocols that employ deep reinforcement learning (DRL) techniques, referred to as deep-reinforcement (CS-DLMA). The goal CS-DLMA is enable efficient and equitable spectrum sharing among group co-located heterogeneous wireless networks. Existing CSMA protocols, such the medium control (MAC) protocol WiFi, are designed for homogeneous network in which all nodes adopt same protocol. Such suffer from severe performance degradation environment where there adopting other MAC protocols. aims circumvent this problem by making use DRL. In particular, adopts ?-fairness general objective CS-DLMA. With ?-fairness, can achieve range different objectives (e.g., maximizing sum throughput, achieving proportional fairness, or max-min fairness) when coexisting with MACs changing value ?. A salient feature it these without knowing through process based on underpinning DRL technique Q-network (DQN). However, conventional DQN algorithms not suitable due their uniform time-step assumption. time steps non-uniform duration required carrier sensing smaller than data transmission. introduces formulation address issue. Our simulation results show TDMA, ALOHA, WiFi adjusting its own transmission strategy. Interestingly, we also find more Pareto e.g., p-persistent CSMA, WiFi. Although focuses our networking, believe domains.